The present invention relates to video image processing, and in particular to processing of an image sequence having consecutive video images in order to improve the spatial resolution of video images having moving content or to reduce motion blur.
The representation of real processes by a video image sequence is inevitably associated with a loss of spatial and temporal resolution. Loss of spatial resolution occurs because only a limited number of pixels are available for representing a video image by display devices such as for example cathode-ray tubes, liquid-crystal displays (LCDs), plasma displays or projection systems. Loss of temporal resolution occurs because a video image sequence constitutes only a temporally discrete sequence of individual images having for example a frequency of 50 Hz or 100 Hz. A viewer is given the impression of a continuous representation because human perception is incapable of temporally resolving image sequences having an image frequency of more than 50 or 60 Hz.
The brightness of an object represented in an image sequence as perceived by a viewer corresponds to the integral of the luminous flux density or intensity of exposure over time. In temporally consecutive video images of the image sequence, an object moving continuously in reality appears at different positions within the individual images. Here human perception integrates the luminous flux intensity not only temporally but also spatially over the direction of motion of the object. The result is a perceived motion blur. Edges of the object that run transversely to the direction of motion and are represented sharply or in high contrast in the individual images appear blurred to the viewer. This blur is more pronounced the faster the object is moving—that is, the greater the distance between positions of the object in consecutive images—and the longer the object is exposed within an image. This effect is less pronounced with cathode-ray tubes, where each pixel of the image tube is exposed or sampled by the electron beam for only a short time, than with LCD screens, where a pixel is exposed for the full duration of an image. The blur effect is described in U.S. Patent Application 20060280249.
A known way of reducing this blurring effect is black frame insertion, that is, the insertion of a black image between two consecutive images. With LCD monitors this can be done by turning off the background illumination in interval-wise fashion. The result, however, is a reduction in image brightness.
The article entitled “Mastering the Moving Image: Refreshing TFT-LCDs at 120 Hz,” by B. W. Lee et al. SID '05, Digest, pages 1583-1585, discloses an image processing technique in which images of a 60 Hz image sequence are displayed repeatedly in order in this way to obtain a 120 Hz image sequence. One of the two like images here is represented brighter than the repeated original image; the other is represented darker than the repeated original image. This method can, however, lead to a perceived motion blur, particularly in the case of bright objects.
Another technique for reducing the blurring effect is to enhance the image resolution by interpolating, in motion-adapted or motion-compensated fashion, one or a plurality of intermediate images between two consecutive images of the initial image sequence. This procedure is described in the article entitled “System-on-Silicon Solution for High Quality HDTV Video Deinterlacing and Frame Rate Conversion for Flat Panel Displays,” by M. Schu et al., ADEAC 2005, Portland, U.S.A., Oct. 25-27, 2005, or in the article entitled “Frame Rate Conversion IC for 120 Hz Flat Panel Displays,” by M. Schu and P. Rieder, Electronic Displays 2007, Nuremberg, Feb. 14-15, 2007.
There is a need for improved processing of an image sequence having consecutive video images, where the processing reduces perceived blurring in the case of moving objects.